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Abstract: Mathematical Morphology proposes to extract shapes from im- 
ages as connected components of level sets. These methods prove very suit- 
able for shape recognition and analysis. We present a method to select 
the perceptually significant (i.e., contrasted) level lines (boundaries of level 
sets), using the Helmholtz principle as first proposed by Desolneux et al. 
Contrarily to the classical formulation by Desolneux et al. where level lines 
must be entirely salient, the proposed method allows to detect partially 
salient level lines, thus resulting in more robust and more stable detec- 
tions. We then tackle the problem of combining two gestalts as a measure 
of saliency and propose a method that reinforces detections. Results in 
natural images show the good performance of the proposed methods. 

Keywords and phrases: topographic maps, level lines, edge detection, 
Helmholtz principle. 

1. Introduction 

Shape plays a key role in our cognitive system: in the perception of shape lies 
the beginning of concept formation. 

Artists have implicitly acknowledged the importance of shapes since the dawn 
of times. Indeed, despite that lines do not divide objects from their background 
in the real world, line drawings are present in much of our earliest recorded art 
and, remarkably, remained unchanged through history, see Figure 1. 

Although art may provide clues to understand shape perception, it tells us 
little from the formal point of view. Let us begin by defining what is a shape. 

Phenomenologists [3] conceive shape as a subset of an image, digital or per- 
ceptual, endowed with some qualities permitting its recognition. In this sense. 
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Fig 1. Lines are used to convey the outer contours of the horses in a very similar way 
in these drawings, one from 15,000 BC (left: Chinese Horse, paleolithic cave painting at 
Lascaux, France) and the other from AD 1300 (right: Jen J en- fa, detail from The Lean 
Horse and the Fat Horse, Peking Museum, China). Reprinted by permission from Macmillan 
Ltd: NATURE [9], copyright 2005. 

both concepts, shape and recognition, are intrinsicahy intertwined: one has to 
define what is a shape in such a way that its recognition can be performed. 

Fohowing these hnes of thought, gestaltists [1] regard shape perception as the 
grasping of structural features found in or imposed upon the stimulus material. 
The Gestalt school has extensively studied phenomena that unveil and justify 
this definition [14, 33]. 

Formally, shapes can be defined by extracting contours from solid objects. In 
this context, shapes are represented and analyzed from an infinite-dimensional 
approach in which a shape is the locus of an infinite number of points [16]. 
This point of view leads to the active contours formulation [15] or to level-sets 
methods [28]. Although these shapes can be defined in any number of dimen- 
sions, e.g. the contour of a three dimensional solid object is a surface, we will 
restrict ourselves to the two dimensional case, following Lisani et al. [17] and 
Cao et al. [5]. 

We define an image as a function : ^ R, where u{x) represents the 
gray level or luminance at point x. Our first task is to extract the topological 
information of an image, independent of the unknown contrast change function 
of the acquisition system. This contrast change function can be modeled as a 
continuous and increasing function g. The observed data of an image u might 
be any such g{u). This simple argument leads to select the level sets [28], or 
level lines, as a complete and contrast-invariant image description [7, 8] . 

Given an image the upper level set Xx and the lower level set of level 
A are subsets of R^ defined by [8] 

A'a = {x G R^ I u{x) > A}, (1) 
A'^ = {x G R^ I u{x) < A}. (2) 

If the image u is lower {resp. upper) semi-continuous, it can be reconstructed 
from the collection of its upper {resp. lower) level sets by using the superposition 
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Fig 2. On the left, original image. On the right, the hierarchical representation of the topo- 
graphic map. 



principle [19]: 



u{x) = sup{A I X G A'a}, 
u{x) = inf{A I X e A'^}. 



(3) 
(4) 



We define the boundaries of the connected components of a level set as a level 
line. 

A gray-level digital image Ud is a discrete function in a rectangular grid that 
takes values in a finite set, typically integer values between and 255. To obtain 
a grid independent representation, we can consider an interpolation u of Ud with 
the desired degree of regularity (i.e., u can be C^, C^, etc.). In this work we use 
bilinear interpolation, in which case the level lines have the following properties: 

• for almost all A, the level lines are closed Jordan curves; 

• by topological inclusion, level lines form a partially ordered set. 

For extracting the level lines of such a bilinearly interpolated image we make use 
of the Fast Level Set Transform (FLST) [23]. Notice that the FLST correctly 
handles singularities such as saddle points. We call this collection of level lines 
(along with their level) a topographic map. 

In general, the topographic map is an infinite set and so only quantized 
grey levels are considered, ensuring that the set is finite. Since the connected 
components of level sets are ordered by the inclusion relation, the topographic 
map may be embedded in a hierarchical representation. To make things simple, 
a level line Li is a descendant of another line Lj in the hierarchy if and only if 
Li is included in the interior of Lj. Figure 2 depicts a simple example. 

The Mathematical Morphology school [19, 28] has extensively studied the 
topographic map and its level sets, producing a whole set of tools for image 
analysis. Smoothing filters, usually described by Partial Differential Equations 
(PDE), can be proven to have an equivalent formulation in terms of iterated 
morphological operators [13]. Hence, edge detectors can then be directly ex- 
pressed by combining these operators. 

The previous requirement leads us to define the set of level lines as a com- 
plete and contrast invariant image representation. In apparent contradiction to 
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this fact, many authors, hke Attneave, argue that "information is concentrated 
along contours (regions where contrast changes abruptly)" [3]. For example, 
edge detectors, from which the most renowned is Canny's [4], rely on this fact. 
In summary, only a subset of the topographic map is necessary to obtain a 
perceptually complete description. 

The search for perceptually important lines will focus on unexpected config- 
urations, rising from the perceptual laws of Gestalt Theory [14, 33]. From an 
algorithmic point of view, the main problem with Gestalt rules is their qualita- 
tive nature. Desolneux et al. [11] developed a detection theory which seeks to 
provide a quantitative assessment of gestalts. This theory is often referred as 
Computational Gestalt and it has been successfully applied to numerous gestalts 
and detection problems [6, 12, 27]. It is primarily based on the Helmholtz prin- 
ciple which states that conspicuous structures may be viewed as exceptions to 
randomness. In this approach, there is no need to characterize the elements 
one wishes to detect but contrarily, the elements one wishes to avoid detect- 
ing, i.e., the background model. When an element sufficiently deviates from the 
background model, it is considered meaningful and thus, detected. 

Within this framework, Desolneux et al. [10] proposed an algorithm to detect 
contrasted level lines in grey level images, called meaningful boundaries. Further 
improvements to this algorithm were proposed by Cao et al. [6]. 

In this work, we build upon these methods, presenting several contributions: 

From global to partial curve saliency. The original meaningful bound- 
aries are totally salient curves (i.e., every point in the curve is salient). We 
propose a modification that allows detecting partially salient curves as mean- 
ingful boundaries. This definition agrees more tightly to the observation that 
pieces of level lines correspond to object contours and also yields more robust 
results. 

An extended definition of saliency. The criterion used to establish saliency 
in the original meaningful boundaries algorithm is contrast. Cao et al. [6] pro- 
posed to determine saliency as a cooperation of two criteria: contrast and regu- 
larity. We study some theoretical and practical issues in their formulation. We 
then present a new formulation in which both aforementioned criteria compete, 
instead of cooperating. It is theoretically sound and yields improved detections, 
with respect to the ones obtained by using only contrast. The previous partial 
curve saliency criterion proves determinant in this new formulation 

Strictly speaking, all the proposed algorithms are only invariant to affine 
contrast changes. This can be easily proven when contrast (i.e., the gradient 
magnitude) is used as the saliency measure [5, Lemma 1, p. 19]. Nevertheless, 
the set of meaningful boundaries is not significantly affected by slight deviations 
from this class of contrast changes. 

As a side note, we point out that there are two remaining steps to address in 
order to develop a complete shape detection system: smoothing, and geometrical 
invariance. Let us briefiy discuss them for the sake of completeness. 
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First, during the acquisition, details much too fine to be perceptuahy relevant 
are introduced. It is necessary to use a suitable filtering mechanism. Invariance 
to these fine details may be handled by an appropriate smoothing procedure, i.e., 
the Affine morphological Scale Space (AMSS) [22] or by a subsequent suitable 
shape description method [30]. 

Second, representations must be invariant to weak projective transforma- 
tions. It can be shown that all planar curves within a large class can be mapped 
arbitrarily close to a circle by projective transformations [2]. Moreover, full 
projective invariance is neither perceptually real (humans have great difficul- 
ties to recognize objects under strong perspective effects) nor computationally 
tractable. In this sense, affine invariance is the most we can impose in practice. 
At the same time, the effect of any optical acquisition system can be modeled 
by a convolution with a smoothing radial kernel. It does not commute with 
projective transformations and must be taken into account in the recognition 
process. A multiscale analysis is the only feasible way to treat it correctly. Both 
concepts, affine invariance and multiscale analysis are consistently integrated in 
the work by Morel and Yu [24]. 

The aforementioned tools that cover these issues can be directly applied to 
the level lines detected by our method. For a wide perspective of the complete 
shape recognition chain see the book by Cao et al. [5]. 

The paper is structured as follows. In Section 2 we recall the definition of 
meaningful boundaries and present a generalization that allows to detect par- 
tially salient curves. In Section 3 we address the combination of contrast and 
regularity for the detection of meaningful boundaries. We conclude in Section 4. 

2. Meaningful Contrasted Boundaries 

Let us begin by formally explaining the meaningful boundaries algorithm by 
Desolneux et al. [10]. 

Let C be a continuous level line of the (bilinearly interpolated) image u. We 
consider a discrete sampling of this curve, and denote it by xq, xi, . . . , Xn-i ^. 
This particular sampling is chosen to ensure that \Du\{xi) and \Du\{xi-^i) are 
statistically independent almost everywhere when pixel values of u are consid- 
ered to be independent The gradient magnitude is computed using a standard 
finite difference scheme on a 2 x 2 neighborhood. 

Notation 1. Let He be the tail histogram of \Du\, defined by 



where Du can be computed by a standard finite differences scheme on a 2 x 2 
neighborhood. 

-•^This corresponds to the following 2 steps: i) The intersection of the continuous level-line 
C with the Qedgels of the image gives a set of m points as explained in [8]. ii) We sample 
n = lm/2\ points by taking one out of every two points 




#{x G U, \Du\{x) > /i} 



(5) 



#{x G u, \Du\{x) > miua^en I 



\Du\{x)r 
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Algorithm 1: Computation of e- meaningful boundaries in image u. 

input : An image u and a scalar e. 
output: A set of closed curves tSres- 

*S ^ FLST(ii); // Compute the set of level lines 

Nil ^ #{5}; 

Compute the tail histogram He of \Du\; 

for C eS do 

Compute the length ^ of C; 
12 ^ min \Du\(x)] 

nfac ^iVn HcifiY^^; 

if nfac < £ then <Sres ^ <Sres U {C} 

return *Sres; 



Definition 1. (Desolneux et al. [10]) Let C be a finite set of Nn level lines ofu. 
A level line C G C is a D MM e -meaningful contrasted boundary (DMM-MCB) 
if 

NFA(C) =^ Nil i?c(min \Du\{x))^l'^ < e (6) 

where I is the length of C. This number is called number of false alarms (NFA) 
ofC. 

Actually, / denotes the Euclidean length of the discrete approximation of C. 
In [5] the authors assume that / = 2n, but we found that this approximation is 
not accurate enough, which leads us to make here the distinction between / and 
2n. 

Algorithm 1 shows a possible procedure to obtain all e-meaningful contrasted 
boundaries. 

Background model. Now we shall check the consistency of Definition 1, 
namely that, in average, no more than e curves are detected by chance. In 
order to make this assertion more precise (in Proposition 1 below) we need to 
define the (a contrario) statistical background model that is used to present 
random input images to the boundary detector. Following [6, 10] we do not 
directly introduce a statistical image model, but we only state the statistical 
properties that each level line C in the input set E of level lines should satisfy. 
The actual shape of the curve does not matter. We only require that a random 
gradient value \Du\{xi) be associated to each of the n regularly sampled points 
xo,xi, . . . of C, that these n random variables be independent, and with 

the same distribution P{\Du\{xi) > ji) = Hc{/J^). 

Proposition 1. The expected number of DMM e -meaningful contrasted bound- 
aries in a random set E of random curves is smaller than e, if E follows the 
above background model. 

We refer to the work by Cao et al. [6] for a complete proof. 
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Fig 3. Conceptual consequence of using the minimum contrast to detect boundaries. The left 
image contains a gray gradient and an uniformly black region on its upper and lower halves 
respectively. The right image is constructed by putting in its upper half the minimum gray 
level on the left imagers upper half. If our perception was tuned to use the minimum contrast 
to detect the boundary between the two regions^ we would perceive that the image on the right 
is as contrasted as the one on the left, which is clearly not the case. 

Proposition 1 allows to interpret the meaningful contrasted curves in Defini- 
tion 1 within a multi-hypothesis testing framework: namely, the curves detected 
on an image u are those that allow to reject the null hypothesis (background 
model) Hq: the values of \Du\ are i.i.d., and follow the same distribution as 
gradient magnitude histogram of the image u itself. 

Definition 1 has some drawbacks. From one side, the use of the minimum or 
any punctual measure, for the case, can be an unstable measure in the presence 
of noise. From the other side, it demands the curve to be not likely entirely 
generated by noise (i.e., well contrasted). We already stated that pieces of level 
lines match object boundaries. Moreover, as seen on Figure 3, the use of the 
minimum contrast seems in contradiction with what we perceive. It is therefore 
too restrictive to impose such a constraint. Since we search for object boundaries, 
we think the natural model is to select level lines that have well contrasted parts. 

2,1. Partially Contrasted Meaningful Boundaries 

In this direction, we propose to modify the definition of the number of false 
alarms of a curve, to support a new model where one detects partially contrasted 
curves. This modification was briefly introduced in [31] and is now explained in 
detail. 

Notation 2. Let xq, xi, . . . , Xn-i denote n points of a curve C of length I. Let 
s be the mean Euclidean distance between neighboring points. For x G C denote 
by Ci (0 < i < n) the contrast at Xi defined by Ci = \Du\{xi). We note by jik 
(0 < k < n) the k-th value of the vector of the values Ci sorted in ascending 
order. 
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For k < N eN and p G [0, 1], let us denote by 



def 



the tail of the binomial law. Desolneux et al. present a thorough study of the 
binomial tail and its use in the detection of geometric structures [11]. 
The regularized incomplete beta function, defined by 

is an interpolation B of the binomial tail to the continuous domain [11]: 

S(n, k; p) = I{p- k,n-k^l) (9) 

where n, /c G M. In the case n and k are natural numbers S(n, k;p) = B{n, k;p). 
Additionally the regularized incomplete beta function can be computed very 
efficiently [26]. 

Following Meinhardt et al. [20], for a given curve the probability under Ho 
that at least k among the n values Cj are greater than /i is given by the tail of 
the binomial law S(n, k; Hc{/J^)). Thus it is interesting, and more convenient, to 
extend this model to the continuous case using the regularized incomplete beta 
function 

S(n • /(5,n), ^ • ^(s,n); ^c(m)) (10) 

where l(^s,n) = ^ and acts as a normalization factor. This represents the proba- 
bility under Ho that, for a curve of length some parts with total length greater 
or equal than /(s,n)(^ ~ ^) have a contrast greater than ja. 

Definition 2. Let C be a finite set of Nu level lines of u. A level line C e C is 
a TMA e-meaningful boundary if 

NFAk(C) = Nil K min B{n • /(2,n) , k • /(2,n) ; H,{^k)) < e (11) 

k<K 

where K is a parameter of the algorithm. This number is called number of false 
alarms (NFA) of C . 

The parameter K controls the number of points that we allow to be likely 
generated by noise, that is, a curve must have no more than K points with a 
"high" probability of belonging to the background model. It is simply chosen as 
a percentile of the total number of points in the curve. The procedure is similar 
to Algorithm 1 but replacing NFA by NFA^- 

As usual. Definition 2 is correct if the following proposition holds. 

Proposition 2. The expected number of TMA e-meaningful boundaries, in a 
finite random set E of random curves is smaller than e. 
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Fig 4. There are 4^4 5004 level lines in the center image of a Gaussian noise with standard 
deviation 50. By setting e = 1000^, DMM-MCB detects one boundary (left detail) and TMA- 
MCB detects two boundaries (left and right details). At e = 1, both methods detect zero 
boundaries. 

This very important proof is given in Appendix A.l to avoid breaking the 
flow of the discussion. 

This new model is an extension of the previous one, since NFAx=i(C) = 
NFA(C). In fact, Definition 2 is no other than a relaxation of Definition 1. 
We should expect to have new detections and to detect the same lines, with 
increased stability. This comes from the fact that several punctual measures are 
used and the minimum is taken over their probability. This was experimentally 
checked and some results can be seen in Section 2.3. 

We apply the DMM-MCB and TMA-MCB algorithms to an image of white 
noise, in order to experimentally check that when e = 1 the number of detections 
is in average lower than 1. This is confirmed in Figure 4, where the number of 
detections is actually zero. Even when e = 1000, the number of detections 
remain very small. 

In [6], other modifications are proposed to the basic meaningful boundaries 
algorithm. On the one hand, meaningfulness is computed locally. We will not 
discuss this further, since we are only interested in the redefinition of the NFA 
and its consequences. In any case, our redefined NFA can also be used in the 
same local detection process. On the other hand, only level lines that remain 
stable across several zoom scalings are detected. The reason behind this ap- 
proach is to counter the effect of small perturbations (i.e., noise) in the image. 
Our scheme handles naturally this effect by minimizing a probability instead of 
a punctual measure. This was confirmed in our experiments where multiscale 
stabilization did not provide any visible improvement. 

2.2. Maximal boundaries 

Because of interpolation, meaningful boundaries usually appear in parallel and 
redundant groups, called bundles. Since the meaningful level lines inherit the 
tree structure of the topographic map, Desolneux et al. [11] use this structure 
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Algorithm 2: Computation of maximal TMA e-meaningful boundaries in 
image u. 

input : An image u, a scalar e an integer K. 
output: A set of closed curves tSres- 

S ^ FLST(n); // Compute the set of level lines 

Nu ^ #{5}; 

Compute the tail histogram He of \Du\] 

Sres ^ 0; 

for C eS do 

Compute the length ^ of C; 
n ^ #{x e C}; 

/ii, . . . , fix ^ the K smallest values of \Du\(x), x G C; 

if nfac < £ then <Sres ^ Sres U {C} 
// Maximality-based pruning: 
repeat 

Find an unexplored monotone section Sm in the level lines tree; 
Cm ^ max nfac; 

for C G *Sm do 
L if C G «Sres cind C / Cm then *Sres ^ S^es \ {C} 

until all monotone sections have been explored-^ 
return *Sres; 



to efficiently remove redundant boundaries. From now on, we work on the tree 
composed only of meaningful boundaries. 

Definition 3. (Monasse and Guichard [23]) A monotone section of a level 
lines tree is a part of a branch such that each node has a unique son and where 
grey level is monotone (no contrast reversal). A maximal monotone section is a 
monotone section which is not strictly included in another one. 

Definition 4. (Desolneux et al. [10]) A meaningful boundary is maximal mean- 
ingful if it has a minimal NFA in a maximal monotone section. 

Algorithm 2 depicts the overall proposed procedure. 

Figure 5 shows an example of the reduction of the number of level lines 
caused by the maximality constraint. Parallel level lines are eliminated, leading 
to "thinner edges ." 

In the following, when we refer to meaningful boundaries, both in its DMM 
or TMA versions, we always compute maximal meaningful boundaries. 

Notice that working with representative curves of monotone sections has some 
well-known dangers for particular configurations that rarely occur in practice. 
For example, if the input image contains successively nested objects of different 
increasing shades of gray, the proposed algorithm will detect only one object of 
each nested set. Other definitions that explore local maxima of some saliency 
measure along the tree, such as MSER [18], can be used to correct this issue. 

Desolneux et al. [10] also proposed an algorithm called meaningful edges 
which aims at detecting salient (i.e., well contrasted) pieces of level lines. TMA- 
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Fig 5. Effect of the maximality condition over the meaningful boundaries of an image. On 
the left, original image; on the center, DMM-MCB (8987 lines found); on the left, maximal 
DMM-MCB (517 lines found). 

MCB can be considered a hybrid of meaningful boundaries and meaningful edges 
and presents advantages from both algorithms. Pieces of level lines belonging to 
different level lines cannot be compared, since they can have different positions 
and lengths. This means that we cannot compute maximal meaningful edges 
in the level lines tree. The TMA-MCB algorithm is able to detect partially 
salient curves while retaining compatibility with the maximality in the tree. 
On the other side, it is possible to compute maximal meaningful edges inside 
a given curve. TMA-MCB, as a provider of the supporting level lines, can be 
considered a first step towards finding meaningful edges that are maximal in 
both directions: in the tree, i.e., orthogonal to the curve, and along the curve. 
The extraction of the optimal pieces in a curve is discussed by Tepper et al. [32]. 

2,3. Practical implications of the change in the NFA 

We now address the following question: is there a fundamental difference in 
practice between DMM-MCB and TMA-MCB? The answer is that, given an 
image, this change implies noticeable differences in the detected curves. Indeed, 
TMA-MCB are more robust since the NFAs attained are much lower. Taking 
the minimum of probabilities is also more stable than taking the minimum on 
any punctual measure, see Figure 6. 

In some cases, by relaxing the meaningfulness threshold in DMM-MCB, that 
is setting £ > 1, visually better results can be achieved. More level lines are kept, 
but at the expense of having lower confidence on them. The key advantage with 
TMA-MCB is that, for a given threshold for less visually salient level lines 
are discarded. 

One of the possible arguments against TMA-MCB could be that it is no more 
than a shift of the threshold on the NFA of DMM-MCB. Specifically, that there 
exists a threshold e' > e for which DMM-MCB and e' would be the same as 
TMA-MCB and e. However, the assertion is clearly false, as shown in Figure 7. 

In many applications (e.g., scene reconstruction, image matching), underde- 
tection is far more dangerous than overdetection. Losing structure is critical 
as it can end-up in a total failure. Detection noise can always be handled (or 
even tolerated) when the amount of noise does not occlude information, as in 
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IMAGE DMM-MCB TMA-MCB 




Fig 6. Noise contamination example. The image on the bottom left is contaminated by a 
small amount of noise. DMM-MCB takes a minimum of punctual measures, thus its result 
is affected. On the counterpart, result with TMA-MCB is less affected, as it deals with prob- 
abilities. Notice that here no smoothing is performed previous to detection, contrarily to the 
original implementation of the meaningful boundaries algorithm [10]. 




Fig 7. Definition 2 is not merely a shift of the threshold on the NFA from Definition 1: even 
relaxing the threshold to its limit (e = 1), the result with the old method remains roughly the 
same. A lot of structure missed with Definition 1 is recovered with Definition 2. 
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Fig 8. Comparison between the stability of DMM-MCB and TMA-MCB. Much lower NFAs 
are attained with the latter in lines which are visually relevant. 

our case. TMA-MCB has an advantage over DMM-MB in this respect^. This is 
experiment ahy checked in ah examples, even if the difference is more striking in 
some examples than in others. 

Figure 8 shows the numerical robustness attained with TMA-MCB. The visu- 
ally important boundaries in the image have a much lower NFA with TMA-MCB 
than with DMM-MCB. 

3. Combining contrast and good continuation 

As already stated, in natural images contrasted boundaries often locally coincide 
with object edges. Thus, they are also incidentally smooth. Active contours [15] 
rely on this combination of good contrast and smoothness to provide well local- 
ized contours. In this section, we reprise the work by Cao et al. [6] and study 
the possible influence of smoothness in the a contrario detection process. We 
conclude that regularity plays an important role in the improvement of the 
quality of the obtained detections. This reinforcement phenomenon and the fact 
that each partial detector can detect most image edges prove a contrario that 
contrast and regularity are not independent in natural images. 

Let C be a rectifiable planar curve, parameterized by its length. Let / be the 
length of C and x = C(r) G C With no loss of generality, we assume that 
r = 0. 

^Note however that overdetection might have as well a huge detrimental impact in other 
applications. 



M. Tepper et al. /Contrast and Regularity in Perceptual Boundary Saliency 



14 



X = C(0) 




Fig 9. Reproduced from the work by Cao et al. [6]. The regularity at x is obtained by comparing 
the radius of the circle with s. The radius is equal to s if and only if the curve is a straight 
line. If the curve has a large curvature, the radius will be small compared to s. 



Definition 5. (Cao et al. [6]) Let s > be a fixed positive value such that 
2s < I. We call regularity of C at x (at scale s) the quantity 

^ _ max{\x - C{-s)\,\x - C{s)\) ^^^^ 
^ s 

where \xi — Xj\ represents the Euclidean distance between Xi and xj. 

Figure 9 visually explains the pertinence of this definition. Only when one of 
the subcurves 5,0)) or C((0,s)) is a line segment, Rs{x) = 1; in all other 
cases Rs{x) < 1. When s is small enough, regularity is inversely proportional to 
the curve's curvature around x [6]. 

The question about the choice of s arises naturally and was studied in detail 
by Cao et al. [6] and Muse [25]. We will limit ourselves to state that a larger value 
of s (thus at less local scale of analysis) is more robust to noise. On the other 
side, s should not be too large either. In practice, and following Cao et al. [6] 
one may safely set 5 = 5, which is the value we use in our experiments. 

Let us denote by Hs{r) the distribution of the regularity in white noise level 
lines, i.e., 

Hs{r) = p(^Rs{x) > r, X G C, C is a white noise level line^, (13) 

which depends only on s and can be empirically estimated. 

Again, the curve detection algorithm consists in adequately rejecting the null 
hypothesis Hq: the values of \Rs\ are i.i.d., extracted from a noise image. We 
assume that, in the background model, contrast and regularity are independent. 

Let us forget for the moment the issues associated with the use of extremal 
(the minimum) statistics, discussed in Section 2. 

Definition 6. Let C be a level line in a finite set C of Nu level lines of image 
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u. Let 

min \Du\{x), 

min Rs(x) 

xec ^ ' 

he respectively the minimal quantized contrast and regularity along C . The level 
line C is a D MM e -meaningful regular boundary (DMM-MRB) if 

NFA^(C) Nil H,{p)"^-' < e. (14) 

The level line C is a DMM e-meaningful contrasted regular boundary (DMM- 
MRB) if 

NFA^^(C) Nu max . HsipY^') < e. (15) 

Remark. Cao et al. [5] provided the following definition of meaningful con- 
trasted regular boundaries: 

NFA^^(C) ^=^' Nu hM^^ H^ipY^^' < e. (16) 

Unfortunately, they do not prove that the expected number of e-meaningful con- 
trasted regular boundaries in a finite set of random curves is smaller than e. 
This fact is annoying since the threshold e is emptied of meaning. R is not by 
any means an easy proof and we have not found a solution yet. However, we 
have proven that by slightly changing their definition in the following manner 

NFACi^(C) Nil H^i^if/'^ H,{pf'^\ (17) 

a proof can be built [29]. 

Although theoretically sound, meaningful contrasted regular boundaries de- 
fined by Equation 17 do not provide satisfactory results. This is a consequence 
of using the exponent P. With respect to DMM-MCB (Definition 1, p. 6) and 
even if the regularity term has high probability (say one), raising the contrast 
term to a much larger power will shift the NFA of all curves towards zero. Irreg- 
ular curves that were not meaningful by their contrast, might become meaningful 
regular boundaries. This is certainly an unwanted side effect. A 

Definition 6 exhibits some interesting properties: 

• A contrasted but irregular curve will not be detected; 

• A regular but non-contrasted curve will not be detected; 

• An irregular and non-contrasted curve will not be detected; 

• A regular and contrasted curve will be detected. 

Both gestalts, i.e., contrast and good continuation, interact in a novel way: 
instead of cooperating by reinforcing each other, as in Equation 17, they compete 
for the "control" of the curve. As the exponent in the contrast term is greater 



p = 
P = 
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than the exponent in the regularity term (/ > l/s)^ the contrast term wih in 
general dominate the detections and the regularity will act as an additional 
sanity check. 

The shifting phenomenon mentioned in the above remark will still be present. 
However, 21 is much less aggressive than and its effect will be doubly mit- 
igated: (1) since / ^ 2 and (2) because of the controlling effect of using the 
maximum. 

Since TMA-MCB is a relaxed version of DMM-MCB, we profit from such 
knowledge and also relax the definition of meaningful contrasted regular bound- 
aries. This relaxation will prove particularly relevant for the contrasted regular 
case. 

Definition 7. Let C be a finite set of Nu level lines of u. A level line C G C is 
a TMA £-meaningful contrasted regular boundary (TMA-MCRB) if 

I min Ic{C,kf\ 

where 

Ic{C,k) = B{n ■ /(2,n),^ • k'^.nY^^cil^k)), 
Is{C,k) = B{n ■ /(2s,n),^ • k2s,ny,Hs{pk)), 

and Kc and Ks are parameters of the algorithm. This number is called number 
of false alarms (NFA) of C . 

Here Kc and Ks have the same meaning as K in Definition 2 and they are 
also set as a percentile of the total number of points in the curve. 

Proposition 3. The expected number of TMA e-meaningful contrasted regular 
boundaries in a finite set E of random curves is smaller than e. 

This very important proof is given in Appendix A. 2 to avoid breaking the 
flow of the discussion. 

For completeness, we provide the following definition. 

Definition 8. Let C be a finite set of Nu level lines of u. A level line C G C is 
a TMA £-meaningful regular boundary (TMA-MRB) if 

NFA^(C) = Nu min B{n ■ l^2s,n),k ■ k2s,n)\ H,{pu)) < e, (19) 

k<Ks 

and Kg is a parameter of the algorithm. This number is called number of false 
alarms (NFA) of C . 

As a sanity check, we apply the DMM-MCRB and TMA-MCRB algorithms 
to an image of white noise. We would expect that when e = 1 the number 
of detections is in average lower than 1. This is checked in Figure 10, where 
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Fig 10. There are 4^45004 level lines in the left image of a Gaussian noise with standard 
deviation 50. By setting e = 1000, DMM-MCRB detects zero boundaries and TMA-MCRB 
detects two boundaries forming a packet (right detail). At e = 1, both methods detect zero 
boundaries. 



the number of detections is actually zero. Even when e = 1000, the number of 
detections remain negligible. 

An immediate objection to the use of regularity might be: since high curvature 
points are often regarded as very meaningful perceptually [3], why such an 
emphasis in discarding them? The answer is also immediate: we detect partially 
contrasted and regular level lines. Hence, a curve containing a relatively small 
number of high curvature points will be detected by TMA-MCRB but not by 
DMM-MCRB. In this scenario, these high curvature points will become more 
surprising, because of their seldomness, and thus meaningful. 

The procedure for finding maximal meaningful regular or contrasted regu- 
lar boundaries is similar to Algorithm 2, replacing NFA by NFA^ or NFA^^, 
respectively. 



3.1, Discussion 

We will now examine the results of the proposed competition between contrast 
and good continuation. 

The benefits of using meaningful contrasted regular boundaries are clear in 
Figure 11. In both examples, only using contrast produces an overdetection 
(level lines are detected in areas with texture, e.g. the vegetation on the left, or 
exhibiting a slight gradient, e.g. the sky and the dome on the right) while only 
using good continuation produces an underdetection (e.g. the bridge on the left 
and the bell on the right). The combination of both gestalts corrects the issues 
by keeping the best from both worlds: most undesired level lines disappear (e.g. 
the vegetation on the left and the sky on the right) while the desired ones are 
kept (e.g. the bridge on the left and the bell on the right). 

Although more complicated to analyze. Figure 12 further supports our claims. 
See the detail on Harrison Ford's sleeve: it is completely lost by using contrast, 
partially recovered by using good continuation and well recovered by combining 
them. 



M. 
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Fig 11. Comparison of TMA-MCB (Definition 2), TMA-MRB (Definition 8), and TMA- 
MCRB (Definition 7). 
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It is important to point out that in general, good continuation has a pre- 
dominant effect over contrast. In the depicted examples, meaningful contrasted 
boundaries have lower NFAs than meaningful smooth ones. This explains the 
visual effect that we perceive when looking at the results: contrasted regular 
boundaries are basically regular boundaries reinforced by some contrasted parts. 

The example in Figure 13 is a real scene, extremely complicated from the 
edge detection point of view. In any case, all results are globally satisfactory. 
Noticeable differences between the methods are perceived by looking at the signs 
containing letters. 

We lastly compare TMA-MCRB with DMM-MCRB in Figure 14. As already 
stated TMA-MCB often detects more structure than DMM-MCB (second and 
third rows). This effect is amplified in DMM-MCRB, and can lead to severe 
underdetections (fourth row). On the other hand, the relaxation present in the 
TMA version allows to recover the structure more faithfully (fifth row), albeit 
some mild overdetections. 

4. Conclusions 

This work presents a novel contribution to the field of image structure retrieval. 
We think that the topographic map is an extremely well suited theoretical frame- 
work to perform that task. Mathematical Morphology has proved this in depth 
and extension with the work it developed. In that direction, we based our work 
on the algorithm called Meaningful Boundaries [11], introducing a few deep 
modifications that help improve the results. 

First, the criterion of meaningfulness was relaxed. In the new definition, a 
level line can have a non-causal piece and still be considered perceptually im- 
portant. We also provide an intuitive parameter that allows to deal with the 
length of that piece. 

Second, we analyze the interaction of two fundamental cues for the percep- 
tion of contours: contrast and regularity. We propose a new way of combining 
these features in which they compete for the control of the boundary saliency. 
Experiments show the suitability of this combination strategy. 

Examples of the resulting image structure retrieval method were presented, 
soundly showing that its theoretical advantages are also validated in practice. 
The proposed method increases significantly the robustness and the stability of 
the detections. 

As a final remark, the maximality constraint presents some issues. All the 
packets of parallel level line pieces are not eliminated by it. The exploration 
of another kind of algorithm based on maximality along the gradient direction 
might help to eliminate this effect [21]. 

Appendix A: Proofs 

A classical lemma will be needed in the following. 
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Lemma 1. Let X be a real random variable. Let F{x) = Pr(X < x) be the 
repartition function of X. Then, for all t G (0, 1)^ 

Pr(F(X) <t) <t. 

In the same way, let H{x) = Pr(X > x). Then for all t G [0, 1]^ 

Vr{H{X) <t)<t. 



A.l. Meaningful Contrasted Boundaries 

This section proves that TMA-MCB (see Definition 2, p. 8) are theoretically 
correct. As usual, being correct means that the following proposition holds. 

Proposition 4. The expected number of TMA e-meaningful boundaries in a 
finite set E of random curves is smaller than e. 

Proof. For this proof we follow the scheme from Proposition 12 in [5]. 

For all /c, let us denote by the random length of the pieces of C such that 
l^^l ^ l^k- From Definition 2, any curve C is £-meaningful if there is at least 
one < k < K such that Nu K B{n • /(2,n)5 ^fc; ^c(/^fe)) < ^- Let us denote by 
E{C^ k) this event and recall that all probabilities are under 1-Lq: 

Pr(^(C, k)) Pr (B{n • /(2,n), i^fe; H,{^k) < vt^) • 




Fig 14. Comparison of DMM-MCB, TMA-MCB, DMM-MCRB, and TMA-MCRB. DMM- 
MCRB may produce severe underdetections . 
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From Lemma 1, we denote 

X = Lk S{x) = B{n ■ /(2,n),^;^c(/i/e)) 

t = Ft{S{X) <t)= Pr(^(C, k)) 

and finally 

The event defined by "C is e-meaningful" is 

0<k<K 

Let us denote by E-^^ the mathematical expectation under Ho- The expected 
number of e-meaningful curves is defined as (XIcgc ^e{C)) where 1a is the 
indicator function of the set A. Then 



\cec J cec cec 



Nu'K 

0<k<K 0<k<K 



: £. 



□ 



A. 2. Meaningful Contrasted Regular Boundaries 

TMA e-meaningful boundaries (see Definition 7, p. 16) are correct is the follow- 
ing proposition holds. 

Proposition 5. The expected number of e -meaningful contrasted regular bound- 
aries, obtained with Definition 1, in a finite random set E of random curves is 
smaller than e. 

Proof The same assumptions from the previous proof hold. 

Let Xi = is meaningful cLud = H^E . Let US dcuotc by E-^Q the mathe- 
matical expectation under 1-Lq. Then 



= 1 k=l k' = l 



(/ n kc ks \\ 
e K^^^X, I iV = n,i^e = ^c,i^. = . (20) 

\i=l k=lk'^l // 



We have assumed that N is independent from the curves and Kc^ Kg are input 
parameters. Th 
Xir=i where 



parameters. Thus, conditionally to N = the law of "^ZiLi ^i is ^he law of 



-^nkcks max(mino<fc<fe^ /c(Ci,/c)2, mino<fc/<fc^ /s(Ci,fe')^)<£* 
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By the linearity of expectation 

/ n kc kg \ / n kc kg \ n kc kg 

\i=lk^lk' = l / \i=lk=lk' = l / i=lk=lk' = l 

Since Yi is a Bernoulli variable, 

/ / o<T<". 

E{Yi) = Pv{Yi = l) = Pv\nk,ks maxl " %,^,.2l<^ 

- (^^^^-- lo<- J < e I ^ I Pr(L. ^ 0. (22) 

Let us finally denote by ai . . . the / independent values of \Du\ and 7i . . . 7z/s 
the l/s independent values of \Rs\- Again, we have assumed that Li is indepen- 
dent of the gradient and regularity distributions in the image. Thus conditionally 
to Li — 

min Ic{Ci^k)^\ 
Ft \ nkc kg max ^ \ < e \ Li = I 

min iJCi.kf^ 

= Pr nkcks max < ^ = 

\ min Is{Ci,k''^ 

\0<k'<kg 

min Ic{Ci,k)\ . . 1/2^ 



Pr I max | / ' ^ 

min Is{Ci,k) 

^0<k'<ks 



= Pr min Ic{Ci, k) < 



X l/2^ 



o<k<kc \nkck. 

Ft I min /,(a,A:0< (- 



e ' ^ 



(23) 



From proof of Proposition 2, 

e X 1/2^ 

Pr ( min Ic{Ci, k) < ' 



, o<fc<fec \nkrk 



Pr min lJCi,k')<{——] < 



\ 1/2 / \ 1/2 

<(— ) (^-] =—. (24) 

Ti kQ kg J y 71 kg J Ti kf^ kg 



M. Tepper et al. /Contrast and Regularity in Perceptual Boundary Saliency 



25 



Finally 

m)<^ ^ EEEe(>^0<^. (25) 

^^^^^ i=ik=ik^^i 

□ 
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